A Unified Framework for Scope Learning via Simplified Shallow Semantic Parsing

s 94.99 94.35 94.67 Papers 90.48 87.47 88.95 Negation cue recognition Clinical 86.81 88.54 87.67 Abstracts 83.74 93.14 88.19 Papers 73.02 82.31 77.39 Speculation cue recognition Clinical 33.33 91.77 48.90s 83.74 93.14 88.19 Papers 73.02 82.31 77.39 Speculation cue recognition Clinical 33.33 91.77 48.90 Table 9: Performance of automatic cue recognition with automatic parse trees on the three subcorpora Table 9 presents the performance of cue recognition achieved with automatic parse trees on the three subcorpora. It shows that: 1) The performance gap of cue recognition between golden parse trees and automatic parse trees on the abstracts subcorpus is not salient (e.g., 95.61 vs. 94.67 in F1-measure for negation cues and 88.49 vs. 88.19 for speculation cues), largely due to the features defined for cue recognition are local and insenstive to syntactic variations. 2) The performance of negation cue recognition is higher than that of speculation cue recognition on all the three subcorpora. This is prabably due to the fact that the collection of negation cue words or phrases is limitted while speculation cue words or phrases are more open. This is illustrated by our statistics that about only 1% and 1% of negation cues in the full papers and the clinical reports subcorpora are absent from the abstracts subcorpus, compared to about 6% and 20% for speculation cues. 3) Unexpected, the recall of speculation cue recognition on the clinical reports subcorpus is very low (i.e., 33.33% in recall measure). This is probably due to the absence of about 20% speculation cues from the training data of the abstracts subcorpus. Moreover, the speculation cue “or”, which accounts for about 24% of specuaiton cues in the clinical reports subcorpus, only acheives about 2% in recall largely due to the errors caused by the classifier trained on the abstracts subcorpus, where only about 11% of words “or” are annotated as speculation cues. Scope Identification with Automatic Cue Recognition Table 10 lists the performance of both negation and speculation scope identification with automatic cues and automatic parse trees. It shows that automatic cue recognition lowers the performance by